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ABSTRACT 

The Department of Defense is developing norms for its 
Armed Services Vocational Aptitude Battery (ASVAB) and the newly 
created Interest Finder, a vocational interest inventory. The 
normative effort, called the 1997 Profile of American Youth (PAY97) , 
will be undertaken as part of the Bureau of Labor Statistics' 1997 
National Longitudinal Survey of Youth. The national norms that result 
from this cooperative project will be used for military personnel 
selection and placement purposes for enlistees and as the basis for 
career selection in high schools in the student testing program. The 
ASVAB will be normed in its recently implemented computerized form, 
and Interest Finder norms will also be developed from its computer 
version. Methodological studies are being planned to address 
substantive issues related to procedures and materials for the 
interview, test administration, and data analysis stages. A 
participation incentive and performance bonus study has preceded 
pilot testing involving more than 1,500 examinees. Other studies that 
are being initiated are studies of the appropriateness of the tests 
with adolescents, a hardware and environment study, and a pretest of 
the longitudinal youth survey. The ASVAB and Interest Finder will be 
administered in the summer of 1997 to the 19,000 subjects of the 
longitudinal study. Normative information will then be available for 
the planned uses of both measures. (Contains one figure, two tables, 
and eight references.) (SLD) 
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Introduction 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



In the latter half of this decade, there are a number of significant improvements 
planned in the Department of Defense’s (DoD) Testing Programs. First, a computerized 
adaptive version of the Armed Services Vocational Aptitude Battery (CAT-ASVAB), that 
currently is operational on a limited basis, will be implemented nationwide in all Military 
Entrance Processing Stations (MBPS). A new vocational interest inventory, the Interest- 
Finder, was introduced in July of 1995 for use in high schools as part of the DoD s 
Student Testing Program. And, the DoD is in the process of developing norms on the 
ASVAB, that was last normed in 1980, and the newly created Interest-Finder. The 
normative effort is called the 1997 Profile of American Youth (PAY97) and will provide 
a profile of the aptitudes and interests of young adults and high school students in this 
country. The PAY97 will be undertaken as part of the Bureau of Labor Statistics’ 1997 
National Longitudinal Survey of Youth (NLSY97) in a cooperative project between the 
Department of Labor (DoL) and the DoD. 

This cooperative effort will serve two important purposes. First, it will provide 
aptitude and interest information on NLSY97 respondents; this information will be useful 
for years to come in analyzing the results of the initial survey and periodic follow-up 
surveys. Second, it will provide nationally representative norms for the ASVAB and the 
Interest-Finder. These norms will be used for military personnel selection and placement 
purposes, and as the basis for career exploration in high schools nationwide in DoD s 
Student Testing Program. 



DoD Testing Programs 

There are two DoD Testing Programs in which ASVAB norms will be developed. 
These programs are the Enlistment Testing Program and the Student Testing Program. In 
the Enlistment Testing Program, the ASVAB is administered to military applicants in 
MEPS where all enlistment screening (medical, aptitude, etc.) is conducted and in Mobile 
Examining Team (MET) sites where there is only ASVAB testing. In the Student Testing 
Program, the ASVAB is administered to students in Grades 10, 11, and 12, and post- 



* Paper prepared for presentation at the annual meeting of the National Council on Measurement in 
Education, New York City, April 1996. 
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secondary schools. ASVAB norms are used in both programs to show each examinee's 
standing relative to a reference population to determine enlistment eligibility 
qualifications and to classify individuals into occupations. The STP also uses separate 
norms by grade and gender to aid career counseling. STP materials help students link 
their interests, aptitudes from the ASVAB, and personal preferences to civilian and 
military occupations. 

At this time the ASVAB is administered primarily in paper-and-pencil (P&P) 
mode in both the Enlistment and Student Testing Programs. However, staff from the 
Navy Personnel Research and Development Center (NPRDC) have developed a 
computerized adaptive version of the ASVAB, CAT-ASVAB, which significantly 
reduces the amount of testing time required to determine the aptitudes of examinees. The 
P&P ASVAB takes about 3 1/2 hours to complete, while the CAT-ASVAB, on average, 
is administered in 1 1/2 hours. The CAT-ASVAB is in use in 5 MEPS and 1 MET site 
and will be implemented in the remaining MEPS starting in the Fall of 1996. 

Norming Instruments 

ASVAB. Because full implementation of the computerized adaptive version of 
the ASVAB in the MEPS will occur in the next year, the ASVAB will be normed in its 
computerized form. Like its P&P counterpart, the CAT-ASVAB is composed of tests 
that measure verbal, quantitative, science and technical, and speeded aptitudes. Table 1 



Table 1. Tests and Numbers of Items on the Computer Adaptive and Paper-and-Pencil 

Versions of the ASVAB 



Number of Items Number of Items 
(P&P ASVAB) (CAT-ASVAB) 



Geperal Science (C$) 


■■ 25 


15 


Arithmetic Reasoning (AR) 


30 


15 


Word Knowledge (WK) 


35 


15 


Paragraph Comprehension (PC) 


15 


10 


Numerical Operations (NO)'*' 


50 


50 


Coding Speed (CS)* 


84 


84 


Auto & Shop Information <AS) 


■ 25 • 


10 


Auto Information (AI) 




Shop Information ($0 




Ih 


Math Knowledge (MK) 


25 


15 


Mechanical Comprehension ^C) 


25 


15 


Electronics Information (El) 
i^emhling Crtijecti ^ 


20 


15 
■ 15- 



* Speeded test 

** Currently non-operational test will be part of norming battery. 



Test Order in 
Norming Battery 

1 

2 

4 

12 

li 

5 

6 

10 
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lists the tests on the P&P and computer adaptive versions of the ASVAB. Note that the 
P&P Auto and Shop Information test is divided into 2 tests (Auto Information and Shop 
Information) on the CAT-ASVAB to ensure the resulting tests are unidimensional and 
therefore item response theory estimation and adaptive testing procedures can be used 
(Prestwood, Vale, Massey, & Welsh, 1985). 

The CAT-ASVAB contains an additional test, a spatial test, that is being 
considered for operational use. A study reviewing ASVAB test content indicated the 
addition of this spatial test. Assembling Objects (AO), would be beneficial due to the 
estimation that it would reduce adverse impact for females seeking to enter some 
occupational specialties. Currently, AO is administered non-adaptively as part of the 
CAT-ASVAB battery. At this time, AO is not used operationally; data on AO are being 
used for research purposes. Plans are to develop item pools for AO so that the test can be 
administered adaptively, thereby the testing time will be reduced significantly from its 
current administration time of about 1/2 hour. Service research on the utility of AO 
scores in the classification of applicants to occupational specialties will determine 
whether the new test will be used operationally in the future. AO will be included in the 
norming battery in case normative information is needed for eventual operational use. 

Given that the norming battery would be one DoD would have to live with for a 
number of years, DoD examined a variety of changes that could improve the battery. One 
such change was the order of the tests. After careful consideration, the DoD decided on a 
new test order for the ASVAB (see the norming test order in Table 1). One test order 
change was to place together, at the front of the battery, all the tests in the Armed Forces 
Qualification Test (AFQT) composite that is used to determine an applicant’s eligibility 
for Service. Thus, Math Knowledge moves after Paragraph Comprehension. Another 
change was to place the speeded tests. Numerical Operations and Coding Speed, at the 
end of the battery. The decision to place these two tests at the end of the battery was 
made because of the high probability the tests may be later dropped from the battery. 
These tests are used in few Service composites, add little in terms of incremental validity 
in predicting training and job performance, and are the most sensitive to changes in 
answer sheets when administered via paper-and-pencil and hardware when administered 
on computer. These tests are very costly in terms of the equating studies that must be 
conducted every time answer sheets or hardware are changed while providing little in 
return in terms of incremental validity. 



Interest-Finder. The Interest-Finder was developed for use in the Student 
Testing Program (it is not used in the Enlistment Testing Program, at this time) and is an 
interest inventory that helps students identify their dominant vocational interest areas 
based on Holland’s Vocational Personality theory (see Wall, Wise, & Baker, in press, for 
details on the development of the Interest-Finder). Holland’s theory promotes six interest 
areas or codes: Realistic, Investigative, Artistic, Social, Enterprising, and Conventional. 
Associated Student Testing Program materials help students link their interests from the 
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Interest-Finder, aptitudes from the ASVAB, and personal preferences to civilian and 
military occupations. 

The Interest-Finder was added to the Student Testing Program in July 1995. It is 
self-administered in P&P form, is self-scored, and takes less than 20 minutes to complete. 
For the norming study, the Interest-Finder will be administered on computer. 

Previous Norming Efforts 

There were 2 previous efforts to develop aptitude norms for military selection and 
classification tests. The first military norming effort occurred in 1944 and the resultant 
score scale is referred to as the 1944 score scale or World War II score scale. The score 
scale was created from the administration of the Army General Classification Test and 
the Navy General Classification Test to male enlisted recruits and commissioned officers 
who were in all Services in 1944. Test data from these two tests were used to create a 
common score scale, the 1944 score scale, for a new joint-service testing battery, the 
Armed Forces Qualification Test (Maier, 1993). 

The other previous military norming effort occurred in 1980 with the 
administration of the ASVAB to an already existing nationally representative sample 
from DoL’s 1979 National Longitudinal Survey of Youth (DoD, 1982). The norming 
effort was called the 1980 Profile of American Youth (PAY80) and was a collaborative 
project between DoL and DoD. PAY80 was an improvement over the 1944 norming in 
that it included the administration of the ASVAB to a nationally representative sample of 
16-23 year old men and women and thus more accurately represented American youth 
than the 1944 male service population. Additionally, oversamples of African-Americans 
and Hispanics in 1980 provided more accurate aptitude data on these minority subgroups. 
The score scale developed from the PAY80 is referred to as the 1980 score scale and its 
creation is fully described in Maier and Sims (1986). 

ASVAB and Interest-Finder Norming 



Contractual Effort 

The PAY97 is part of a larger effort, the NLSY97. The purpose of the NLSY97 is 
to understand what makes for a successful transition into the workforce and adulthood. 
The NLSY97 involves conducting multiple interviews of thousands of young people over 
several years. Major topics on the interviews include questions about family background, 
schooling, job training, marriage and fertility, health, pro-social and anti-social behavior, 
income and assets, environmental variables, and cognitive measures. The dataset that 
will emanate from the NLSY97 will be one of the richest datasets on labor market trends 
in this country. 

A $29 million 5-year contract to conduct the joint DoL and DoD efforts was 
awarded by DoL in January 1995 to the National Opinion Research Center (NORC). 
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NORC and its subcontractors, the Human Resources Research Organization (HumRRO), 
the Educational Testing Service (ETS), and the Sylvan Technology Centers (STC), are 
involved in the DoD norming tasks. NORC responsibilities include overall management 
of the contract, design of the sampling plan, development of the norming samples, 
development of norms for the Interest-Finder, and participation in the analysis of the 
CAT-ASVAB data. ETS along with HumRRO and STC are responsible for the design 
and conduct of a pilot test of the test administration procedures. ETS with STC will have 
primary responsibility for administering the CAT-ASVAB and Interest-Finder in all 
methodological and pilot studies and the main norming study. The instruments will be 
administered at permanent STC sites and other sites set-up temporarily for the study/ies. 
As part of the main norming study data collection, STC will administer the CAT-ASVAB 
at about 120 of its 250 sites across the country. In addition, ETS will operate about 80 
temporary testing sites for those locations where sample members exist but there is no 
existing Sylvan site. ETS also will conduct analyses of the CAT-ASVAB norming data. 
HumRRO is responsible for conducting various methodological studies to resolve some 
substantive and methodological issues related to the recruiting of participants and the test 
administration procedures. Specifically, HumRRO will research 1) the effects of various 
incentives to increase participation in the study and to motivate test performance once an 
examinee shows up for testing, 2) the effects operational vs. norming conditions may 
have on test scores, and 3) the appropriateness of the CAT-ASVAB and Interest-Finder 
for the very youngest examinees in the samples. 

HumRRO also manages a Norming Advisory Group that reviews the technical 
aspects of the sampling design, methodological and pilot studies, and norm data 
collection and development. The Norming Advisory Group is a panel of five nationally 
recognized experts in one or more of the following areas: test and norm development, 
test equating, computerized testing, item response theory, interest measurement, statistics, 
sampling, and survey methodology. Norming Advisory Group members include Drs. 
Fritz Drasgow and Lenore Harmon from the University of Illinois, Dr. Michael Kolen 
from the American College Testing Program, Dr. Ben King from the Florida Atlantic 
University, and Dr. Norm Bradbum from the University of Chicago. 

In the joint DoL-DoD project, DoD is involved in 4 tasks. The first task involves 
the design and development of nationally representative samples for the PAY97 and 
NLSY97. This includes the development of procedures to maximize participation, 
materials to provide information about the PAY97 and NLSY97, and a questioimaire to 
identify respondents who are eligible for the various samples. In the second task, various 
methodological and procedural issues are examined prior to the third task when the 
norming administration of the ASVAB and Interest-Finder occurs. Finally, the last task 
is the analysis of the aptitude and interest data and the reporting of the results to all 
participants. 
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Sample Design and Development 

Objectives of Norms. The first step in planning the design of the sample was to 
identify the norming objectives. One major objective of ASVAB norms is to fulfill the 
legal requirement which prohibits the enlistment of individuals below the tenth percentile 
on the AFQT, a measure of verbal and math abilities. AFQT scores are reported as 
percentiles and are divided into the categories presented in Table 2. To meet the legal 
requirement and to facilitate annual reporting to Congress on recruit quality, the Services 
use the AFQT to determine whether an applicant meets minimum qualification standards. 
Also, each of the Services uses the AFQT to determine whether applicants qualify as 
"qu al ity" rccruits (high school graduates who score at or above the 50th percentile, AFQT 
Category IIIA and above). Some of the Services provide enlistment bonuses to quality 
recruits who enlist in certain occupations. Therefore, the accuracy of new ASV AB norms 
is imperative and could affect costs associated with bonuses, attrition, and training and 
job performance. 



Table 2. AFQT Percentile Categories* 



AFQT** Category 
I 

IIIA 

DIB 

IV 



Perc^ntiU Range 
93-99 
^ 5-92 
50-64 
31 -49 
10-30 
1-9 



* By law, those in AFQT Category V are precluded from 
enlistment and percentage of AFQT Category IV recruits 
is limited. 

** AFQT = 2(VE) + AR + MK, where VE = WK + PC 



A second objective is to provide aptitude information for use in the classification 
of military applicants to specific occupational specialties. Based on Service-specific 
studies analyzing the utility of the ASVAB to predict training and on-the-job 
performance, each of the Services defines composites using different ASVAB tests. The 
Services determine the cut scores on the composites, above which an applicant must score 
to be considered for a particular occupation. These first two objectives (to meet AFQT 
legal requirements and provide classification information) would require both accurate 
and up-to-date normative information on American youth that meet general enlistment 
eligibility standards based on age. Also, accurate aptitude information on American 
youth would aid the evaluation of the impact of different selection and classification 
policies on different gender, race, and ethnic groups. 
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A final objective for ASVAB norms applies only to the Student Testing Program. 
In addition to meeting the objectives discussed previously, noims provide participants in 
the Student Testing Program with accurate information on their standing relative to their 
peers in the skills measured by the ASVAB and also in the interests measured by the 
Interest-Finder for the purposes of facilitating career exploration. 

Sample Development. There will be three samples developed for DoD and DoL 
purposes. Each of these samples will be representative of the non-institutionalized 
population of the United States in terms of gender, socioeconomic status, ethnicity /race, 
geographic region, and an urban/suburban/rural indicator. One sample will be for the 
NLSY97 effort and will consist of 12,000 respondents who are 12-17 years of age. The 
other two samples are for use in the PAY97 and are defined by the norming objectives for 
the ETP and STP. 

For the selection and classification objectives of ASVAB norms for the ETP, the 
appropriate normative population is enlistment eligible American youth. To obtain 
updated information on the aptitudes of potential applicants, one of DoD s samples, the 
ETP sample, will consist of 6,000 18-23 year olds. To allow analysis of different 
selection and job assignment policies for different subgroups, the ETP sample is designed 
to contain sufficient numbers of minorities (i.e., females, Hispanics, and non-Hispanic 
Blacks) to provide accurate aptitude estimates of these groups. Specifically, it is 
estimated that 1,500 per subgroup are required to provide accurate estimates. 

The final sample will be used to develop STP norms. The norming objective for 
the STP is to provide career guidance information to students in Grades 10, 1 1, 12, and 
post secondary school. In the STP, aptitude and interest scores are reported to students 
for their same gender/same grade and opposite gender/same grade. To provide accurate 
norms, approximately 1,100 examinees are required for each of these groups for a total of 
6,600 students in Grades 10, 11, and 12. Given the sample definitions, the STP and DoL 
samples will overlap considerably and there will be less overlap between the STP and 
ETP samples and ETP and DoL samples. Figure 1 shows the overlap between the 
samples. In all, it is estimated that about 19,000 subjects will be required for both the 
NLSY97 and PAY97. 

To develop the DoD and DoL samples, a house to house survey will be conducted 
during February - May 1997. Work has already begun on identifying those households 
that will be contacted to determine if there are eligible participants for the NLSY97 and 
PAY97. The first step in the process was to identify primary sampling units (PSUs) for 
two samples. One sample is designated a “cross-section” and is a nationally 
representative sample of PSUs. The other is a supplemental sample and consists of PSUs 
that have high concentrations of Blacks and Hispanics. The supplemental sample is 
needed to obtain the required numbers of Blacks and Hispanics in these subgroups. One 
hundred PSUs for the cross-section and the supplemental sample have been identified for 
a total of two hundred PSUs. Neighborhoods (one or more adjoining blocks) within each 
of the PSUs have also been selected. The next step is to list all the households within 
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Figure 1. NLSY97 and PAY97 Sample Sizes and Overlap 




each neighborhood and to enter each address. Of particular concern at this stage is to 
ensure that no households are missed and to identify any households that no longer exist 
and any new construction that may have taken place. A total of almost 90,000 
households will be selected for screening; 58,820 of these households will be identified 
through the cross-section and 29,030 through the supplemental sample. 

Sample Screening and Identification. Field staff will conduct CAPl (computer 
assisted personal interview) administered interviews to determine if individuals in the 
households are eligible for any of the DoD or DoL samples. Since the screener questions 
will be asked of everyone in the household, the set of questions must be kept at a 
minimum (only what is necessary to determine sample eligibility and to evaluate any 
nonresponse bias) to keep costs down. Screener items include questions on age, gender, 
race/ethnicity, and grade of education, for example. Additionally, every effort will be 
made to identify individuals who may be connected to a household, but for some reason 
are not at the house. Such individuals include people who are in school dormitories, jail, 
military barracks, or other institutions. Some of these individuals will be considered 
eligible for one or more of the samples (for example, individuals in dormitories will be 
included). Once an individual is identified as eligible for the DoL sample, s/he will 
participate in a lengthier interview (1 hour) as part of the NLSY97 in what is termed a 
“screen and go” procedure. STP and ETP sample participants that do not overlap with 
the DoL sample will not be part of the NLSY97 interview. Since DoL is interested in 
obtaining aptitude information on its sample, eligible respondents for all three samples 
will be asked to participate in the CAT-ASVAB and Interest-Finder administration. To 
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maximize participation in the administration, respondents will be offered a monetary 
incentive that they will receive at the time of testing plus a nonmonetary incentive of their 
test scores along with interpretive materials that will be mailed to them after testing. 

Methodological and Pilot Studies 

A summary of the issues to be researched in the methodological studies will be 
provided here. For further details, Dr. James McBride and Dr. Stawarski in their 
presentations during this symposium will provide the plans and progress of two of the 
methodological studies (McBride & Waters, 1996; Stawarski, 1996). Methodological 
studies are planned to address substantive issues related to various procedures and 
materials that will be used during the interview, test administration, and data analysis 
stages. 



Participation Incentive and Performance Bonus Study. First, to minimize any 
bias that may be introduced due to nonresponse, participation in the CAT-ASVAB and 
Interest-Finder administration must be maximized. One way of enticing people to 
participate is to provide them with a monetary incentive; however, the question is what 
level of a monetary incentive is necessary to obtain a particular participation rate? One of 
the methodological studies, for which data collection was completed recently, was 
designed to address this issue and examined varying levels of monetary incentives. A 
second issue is related to motivating examinees to try their hardest on the aptitude portion 
of the test. Therefore, a second part of the methodological study of incentives was to 
look at the effects of different levels of monetary bonuses on performance. The question 
to be answered is “Does a bonus for performance increase test score performance as 
compared to test scores of those who are not offered a bonus?” Given that there may be 
an additive effect on participation of the total amount that a person is offered (incentive 
plus bonus), participation incentives were crossed with performance bonuses in the study 
design to determine if there is any interaction between these two variables. 

Pilot Study. In conjunction with the participation incentive and performance 
bonus study, a pilot test of administration procedures was conducted. These procedures 
included test administrator training, logistics of getting examinees to the testing facilities, 
test administration procedures, etc. The CAT-ASVAB and Interest-Finder were 
administered to over 1,500 examinees at 19 permanent STC sites and 2 temporary ETS 
sites as part of the combined studies. In relation to the pilot, a few procedures were 
identified as needing improvement. These included 1) communications between STC 
headquarters and the STC testing centers - messages on procedures sometimes were lost 
or not transmitted, 2) testing hours at the centers -- hours need to be expanded to better 
accommodate the examinees, 3) scheduling examinees - scheduling through a second 
party was inefficient and more direct involvement of the examinee in the scheduling 
needs to be accomplished, 4) maps and instructions to the STC testing centers -- maps 
and instructions were not provided, at first, thus some examinees could not find the 
centers, and 5) reminder phone calls - although reminder phone calls were made, calls 
closer to the testing time and more frequent could help increase the rate at which 
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examinees show up for the test. (Kaplowitz, Tysl, McBride, Hedberg, Rock, & Quenette, 
in press). 

Appropriateness Study. An additional methodological study initiated during the 
summer of 1995 is a study to evaluate the appropriateness of the CAT-ASVAB and 
Interest-Finder for 12-14 year-olds. The CAT-ASVAB and Interest-Finder were designed 
for use with examinees in grades 10 and higher; but participants in the NLSY97, from 
whom DoL wants to obtain aptitude and interest information, will be as young as age 12. 
Therefore, it must be determined whether the 2 DoD instruments or some subset of them 
are appropriate for the youngest NLSY97 examinees. Two waves of testing have been 
completed for this study — a total of 60 12-14 year olds were administered the full CAT- 
ASVAB battery and Interest-Finder. Dr. Stawarski will report results of this study and 
plans for the last wave of testing of this age group. 

Hardware and Environment Study. Another methodological study will be 
designed to determine if there are effects in test scores due to differences in test 
administration conditions between the norming conditions and conditions in the 
Enlistment and Student Testing Programs. Both the hardware and the testing 
environments are different between the STC permanent sites and ETS temporary sites and 
the Enlistment Testing Program testing sites (MEPS and MET sites) and Student Testing 
Program sites (high schools). The hardware that will be used in the norming study meets 
the same minimum requirements as the hardware that will be purchased for 
implementation in the MEPS; however, the impact of potential differences between the 
hardware needs to be explored. A recent study by NPRDC showed that hardware 
differences had an effect on test scores for the speeded tests. There were no differences 
in test scores for different desktops; differences were observed between desktops and 
notebooks. 

NLSY97 Pretest. DoL will conduct a pretest of its 1997 longitudinal youth 
survey during the Fall of 1996. The purpose of the pretest is to provide a final check of 
the survey software and to determine the amount of time it will take to complete various 
sections. As part of the NLSY97 pretest, DoD plans to use this opportunity to evaluate 
any remaining methodological issues. 

Because of the differences between the recruiting procedures that were used in the 
Participation Incentive and Performance Bonus study (telephone contacts) versus what 
will be used in the main NLSY97 and PAY97 effort (door-to-door surveys), DoD will 
conduct a Demonstration (of the PAY97) Study as part of the NLSY97 pretest. This 
Demonstration Study would mimic the PAY97 by enlisting the participation of the 12-23 
year-olds located during the pretest in taking the CAT-ASVAB and Interest-Finder. The 
study will attempt to duplicate every detail of the PAY97 main study and will 
“demonstrate” the participation rate that can be expected in the main study. Additionally, 
current plans are to examine two incentive/bonus conditions in the Demonstration Study 
to provide a better evaluation of the effects of a bonus on test score performance. The 
conditions that will be used in the Demonstration Study will be determined from the 
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results of the Participation Incentive and Performance Bonus Study. Lastly, the NLSY97 
pretest will enable DoD to conduct the third and final wave of the Appropriateness Study. 

Administration Plans 

The CAT-ASVAB and Interest-Finder will be administered during the summer of 
1997 to all 19,000 subjects in the NLSY97 and PAY97 at permanent STC sites and 
temporary sites that will be set up and operated by ETS. To avoid any bias in the norms 
based on differences in hardware, the computers at all of the sites meet the same 
minimum specifications -- each are 486 IBM-compatible desktop machines with monitors 
that are comparable. Standard keyboards specially designed to limit the number of keys 
available to the examinee (response keys for 5 options, a help key, and an enter key) were 
purchased for the study and are all the same. In addition to the CAT-ASVAB and 
Interest-Finder, demographic questions and a short survey of the testing experience will 
be asked on-line of each examinee. The total administration time should not exceed 4 
hours and, on average, is expected to be less than 2 hours. 

For the youngest examinees, the 12-14 year olds, the CAT-ASVAB 
administration will be different than for the other examinees. The 12-14 year olds will 
begin each CAT-ASVAB test at an initial item of easier item difficulty. The initial item 
has been adjusted to about 1 standard deviation below the starting point of the operational 
CAT-ASVAB. This change was made so that the testing experience will be less 
frustrating for these examinees. Based on the appropriateness study, additional changes 
to the CAT-ASVAB may be made for these examinees. It is anticipated that some of the 
tests (perhaps the technical tests. Auto Information, Shop Information, Mechanical 
Comprehension, and Electronics Information, and maybe others) will be too difficult for 
the 12-14 year olds and therefore may be dropped for this group. 

After testing, as each examinee checks out of the testing center s/he will receive a 
check as part of the participation incentive. A couple of months after testing, examinees 
will receive their aptitude and interest scores, materials they may use to learn more about 
their scores, and information about educational and career opportunities. The scores will 
be reported relative to the examinee’s norming population (students in Grades 10, 1 1, or 
12; 18-23 year olds; or children who are 12, 13, or 14). Individuals who are administered 
imder nonstandardized conditions will receive scores along with cautions in interpreting 
them. In addition, if the decision is made to provide a performance bonus, the examinees 
will receive a check for their bonus if they answered correctly more than one-quarter of 
the aptitude items. The one-quarter criterion was selected based on simulation studies 
demonstrating that most examinees in a simulated C AT-AS V AB experience would 
answer correctly this percentage of items. 

Data Analysis and Reporting Plans 

For the Enlistment Testing Program and Student Testing Program data, the 
analyses will include computing the percentile rankings and standard scores for each test 
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in the CAT-ASVAB and for each interest code for each normative sample (enlistment 
eligible youth; students in Grades 10, 11, and 12). To get to the point where percentiles 
can be computed will require 1) data editing, 2) tests of sampling assumptions, 3) 
application of missing data algorithms where necessary, and 4) the development of 
weights that will make the samples representative of the population. Additionally for the 
CAT-ASVAB data, models will need to be applied to estimate item and ability 
parameters and methods will be used for linking the parameters onto a common scale. 
Then the representative (weighted) distribution of thetas will be used to compute 
percentile rankings. Data for examinees who test under nonstandardized conditions will 
not be included in the development of either the Student or Enlistment Testing Program 
norms. Analyses and reporting of the results are estimated to be completed in the spring 
of 1998. Implementation of the new normative information of the CAT-ASVAB and 
Interest-Finder is scheduled for January 2000 in the ETP and July 2000 in the STP. 
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